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CHAPTER  I 


INTRODUCTION 

During  the  past  few  decades  the  rate  of  technological  develop- 
ment has  increased  enormously,  causing  the  development  and  use  of 
increasingly  complex  equipment.  This  increased  complexity  of  equip- 
ment, along  with  its  resulting  greater  costs,  has  spurred  interest  and 
developments  in  the  field  of  reliability,  not  only  in  developing  meth- 
ods of  establishing  reliability  requirements  in  the  design  stage  of  an 
item,  but  also  in  developing  reliability  testing  techniques  used  to 
establish  that  the  item  meets  its  reliability  requirements. 

Reliability  testing  procedures  can  be  expensive:  environmental 
requirements,  personnel,  test  instruments,  and  the  value  of  the  items 
put  on  test  all  contribute  to  the  expense.  An  expense  of  testing 
which  is  sometimes  neglected  is  the  cost  of  the  decision  resulting 
from  the  test.  As  an  example,  if  the  military  accepts  a weapon  with 
reliability  somewhat  less  than  what  is  indicated  by  a reliability  test, 
then  the  cost  of  this  incorrect  decision  includes  increased  mainte- 
nance time,  repair  time,  downtime,  and  a decreased  useful  life.  In 
developing  an  economic  testing  procedure,  the  direct  testing  costs 
should  be  balanced  against  the  cost  of  an  incorrect  decision  resulting 
from  the  test. 

The  purpose  of  this  report  is  to  develop  a reliability  testing 
procedure  for  use  by  the  Army  on  prototype  small  caliber  arms  to 
determine  whether  they  meet  the  specified  reliability  requirements. 
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The  testing  procedure  developed  Is  Intended  to  be  cost  effective;  that 
is,  it  seeks  to  minimize  the  total  cost  of  testing,  both  direct  test" 
ing  costs  and  the  costs  resulting  from  the  decision  made  on  the  basis 
of  the  test.  The  prototype  arms  being  considered  are  submitted  to  the 
Army  by  several  private  contractors.  These  weapons  then  undergo  engi- 
neering design  tests  by  the  Army,  and,  depending  on  the  results  of 
these  tests,  one  or  two  of  the  prototypes  are  approved  for  further 
development  and  eventually  production.  Reliability  is  one  of  several 
criteria  used  to  determine  which  weapons  will  continue  development  and 
which  ones  will  be  rejected.  It  can  be  easily  seen  that  the  decisions 
made  at  this  stage  of  a weapon's  development  will  have  a tremendous 
effect  throughout  the  remainder  of  its  life  cycle.  The  discussion 
following  in  Chapter  II  is  a brief  summary  of  the  general  theory  of 
reliability  applicable  to  this  problem.  Chapter  III  presents  a broad 
consideration  of  the  various  types  of  reliability  testing  procedures 
found  in  the  literature.  Chapter  IV  describes  and  evaluates  the 
present  procedures  used  by  the  Army  in  their  engineering  design  tests 
on  prototype  small  caliber  arms,  and  indicates  why  the  present  methods 
used  to  test  reliability  are  unsatisfactory.  Chapter  V further  dis- 
cusses two  types  of  reliability  testing,  truncated  and  sequential 
testing,  and  describes  how  they  both  may  be  applicable  to  the  problem 
considered  here.  Chapter  VI  presents  the  recommended  testing  proce- 
dure, along  with  a numerical  example  of  how  it  can  be  applied.  Final- 
ly, Chapter  VII  contains  some  concluding  remarks,  as  well  as,  a dis- 
cussion of  difficulties  which  may  be  encountered  in  implementing  the 
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proposed  procedure.  Some  recommendations  for  areas  of  further  re- 
search pertaining  to  reliability  testing  of  prototype  armament  are 
also  suggested. 


CHAPTER  II 


I 


I 


BASIC  ASPECTS  OF  RELIABILITY  TESTING 

An  essential  element  in  the  general  growth  of  reliability  acti- 
vities in  the  past  few  years  has  been  the  development  of  various 
statistical  reliability  testing  plans.  Standard  plans  have  been  de- 
veloped from  essentially  the  same  mathematical  framework  with  differ- 
ent underlying  assumptions  and  are  basically  similar  procedurally, 
but  have  been  cataloged  in  many  different  ways.  A specific  plan  can 
be  selected  by  selecting  various  sets  of  test  parameters  such  as  total 
testing  time,  number  of  items  on  test,  maximum  number  of  failures, 
etc.  This  chapter  will  present  a brief  summary  of  the  general  theory 
applicable  to  reliability  tests. 

Reliability  is  defined  in  the  literature  in  various  ways,  but  it 
is  always  expressed  in  some  manner  as  the  probability  of  performing 
without  failure  for  a specified  time  under  given  conditions.  In  the 
statistical  development  of  sampling  plans  the  essential  element  in 
the  definition  is  the  fact  that  reliability  is  expressed  as  the  prob- 
ability of  successful  operation.  What  constitutes  a failure  and  what 
the  given  conditions  are  must  be  specified.  Plans  have  been  developed 
for  both  cases  in  which  time  is  and  is  not  a criterion  of  successful 
operation. 

Mathematically,  then,  the  reliability  function  R can  be  express- 


ed as 
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R = PlX^XfXg)  = F(X2»6i»02»***)  ~ P(Xj »8i»82» • • • ) 

= R(8i »02» • • • ) (2.1) 

where  F(X;e1,e2». . . ) is  the  cumulative  distribution  function  of  the 
random  variable  of  interest  X,  X^,  and  X2  are  the  limits  which  define 
success,  and  0i,e2 »<■-..  are  the  parameters  of  the  distribution  of  X. 

In  the  case  where  success  is  a simple  attribute,  the  parameter  of  the 
reliability  function  is  simply  the  probability  of  success  and 

R » R(e)  = e = probability  of  success.  (2.2) 

Most  reliability  testing  plans  are  based  on  a definition  of 
success  which  uses  a time  period  of  successful  operation.  In  these 
cases  reliability  is  a function  of  the  required  time  of  operation,  t, 
and  the  parameters  0i,e2,...  of  the  distribution  of  the  time  to  fail- 
ure, T.  If  the  distribution  of  the  random  variable,  time  to  failure, 
is  known  then  the  reliability  function  can  be  expressed  precisely. 
Most  reliability  testing  plans  are  based  on  the  Weibull,  gamma,  or  a 
special  case  of  both,  the  exponential  distribution.  The  reliability 
function  for  the  Weibull  distribution  is  given  by 

00  g 

R = R(t;cc,e)  = P(T>t)  = / ax°~1e"*‘<  /6dx  t,o,8>0  (2.3) 

t 

= 0 elsewhere; 

for  the  gamma  distribution  by 

R = R(t;o,e)  « P(T>t)  =/  dx  t,d,8>0  (2.4) 

* ' (e) 

= 0 elsewhere; 

where  a is  equal  to  the  shape  parameter  of  the  distribution, 
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and  for  the  exponential  distribution  by 

R = R(t;e)  = P{T>t}  = fVx/e  dx  = eTt/0  t,0>O  (2.5) 

t 6 

= 0 elsewhere. 

Because  of  the  random  nature  of  the  variable  of  interest,  reli- 
ability cannot  be  measured  directly  but  rather  can  only  be  estimated. 
An  estimate  which  is  good  in  a statistical  sense  will  have  such  prop- 
erties as  being  maximum  likelihood,  unbiased,  minimum  variance,  effi- 
cient, and  sufficient.  For  example,  let  n items  be  placed  on  test, 
where  all  are  tested  to  failure.  Let  t^  be  the  observed  operating 

1L 

time  to  failure  for  the  i item.  The  maximum  likelihood  estimate  of 

the  parameter  0 is,  for  the  exponential  case, 
a n 

0 = (1/n)  z t.,  (2.6) 

i=l  1 

for  the  one  parameter  Wei bull  distribution 

0 = (1/n)  z t.“,  * (2.7) 

1-1  1 

and  for  the  gamma  distribution 
a n 

0 = (l/«n)  l t.,  (2.8) 

1=1  1 

where  a is  equal  to  the  shape  parameter  of  the  distribution. 

Estimates  can  also  be  derived  for  the  case  when  the  test  is 

1L 

terminated  at  the  rtn  failure,  where  r<n,  and  for  a test  with  re- 
placement; that  is,  when  each  item  which  fails  is  either  replaced  or 

1L 

repaired,  and  the  test  is  terminated  at  either  the  r n failure  or  else 
at  a preassigned  time  T.  For  practical  use  a more  meaningful 


Indication  of  the  reliability  of  an  item  is  the' confidence  interval 
estimate.  This  form  of  an  estimate  provides  a range  of  values  which 
contains  the  actual  reliability,  {R^<R<RU>,  with  a specified  degree 
of  confidence  that  R lies  in  this  range.  Usually  a confidence  state- 
ment is  of  the  form  P(R1<R<RU)  = y , where y is  the  confidence  level 
of  the  statement.  Many  times  it  will  specify  only  the  lower,  one- 
sided confidence  interval,  i.e.,  P(Rj<R)  = y.  Lower  confidence  inter- 
val estimates  are  relatively  easy  to  compute  when  only  one  parameter 
is  unknown  and  the  reliability  is  a function  of  only  the  unknown  para- 
meter, as  is  the  case  with  the  exponential  and  one  parameter  Wei bull 
and  gamma  distributions.  In  the  cases  where  more  than  one  parameter 
is  unknown  exact  confidence  limits  are  not  readily  obtained  and  one 
must  normally  resort  to  approximations.  Lloyd  and  Lipow  (15)  , et  al., 
indicate  methods  for  computing  approximate  limits  for  the  gamma  and 
Wei  bull  distributions  where  two  parameters  are  unknown. 

In  practice,  reliability  tests  are  usually  conducted  to  determine 
whether  the  true  reliability  exceeds  some  specified  lower  value  rather 
than  to  make  a specific  estimate  of  the  value  of  the  reliability.  In 
these  cases  it  is  more  appropriate  to  use  hypothesis  testing  proce- 
dures rather  than  confidence  interval  methods  to  obtain  the  desired 
results. 

Hypothesis  tests  can  usually  be  attained  directly  from  a know- 
ledge of  the  form  of  the  confidence  interval.  Suppose  it  is  desired 

* Numbers  in  parentheses  refer  to  numbered  references  in  the  List  of 
References . 
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to  test  the  hypothesis  that  the  actual  reliability  R is  at  least  some 
specified  value  Rg  at  a level  of  significance  a,  where  a is  defined 
by  Ptconcluding  that  R<R$  when  R actually  equals  Rs>  = a.  A suitable 
procedure  is  to  conduct  the  test,  compute  the  (1-a)  upper  confidence 
limit  Ru,  compare  with  the  specified  minimum  value  Rg,  and  make  a 
decision  according  to  the  following  rule:  If  R$  l Ru»  conclude 
Rs  £ R;  but  if  Ry  < R$,  reject  the  hypothesis  that  RS=R  and  conclude 
that  R<Rg.  In  general,  a hypothesis  is  accepted  if  the  hypothesized 
value  lies  in  the  confidence  interval,  as  demonstrated  by  Dellin- 
ger (5). 

In  statistical  sampling  the  differentiating  ability  of  a test 
is  of  paramount  importance  and  is  indicated  by  the  operating  charac- 
teristic curves,  usually  called  O.C.  curves.  These  curves  are  plots 
of  the  probability  of  accepting  a hypothesis  versus  the  actual  value 
of  the  hypothesized  quantity  of  interest.  Figure  1 shows  a typical 
set  of  O.C.  curves  plotted  for  various  sample  sizes  n where  the  quan- 
tity of  interest  is  reliability. 

The  O.C. -curves  can  be  a very  useful  tool  in  selection  of  or 
design  of  a test.  Not  only  does  it  indicate  the  differentiating 
ability  of  a particular  test,  but  also  helps  to  determine  the  risk  of 
drawing  erroneous  conclusions.  As  indicated  by  Figure  1 (page  9), 
increased  sample  size  also  increases  the  differentiating  ability  of 
a test,  but  the  increased  sample  size  also  increases  the  testing  cost. 
There  must  be  a trade-off  between  the  ability  of  a test  to  differen- 
tiate against  an  undesired  level  of  reliability  and  the  cost  of 


Figure  1 - General  form  of  the  operating  characteristic  curves 
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testing. 

A test  can  be  completely  specified  by  the  selection  of  one  point 
on  the  O.C.  curve  and  the  sample  size,  or  by  selecting  two  points  on 
the  O.C.  curve.  Dellinger  (5),  et  al.,  recommends  determining  the  test 
by  selection  of  two  points  on  the  O.C.  curve.  One  point  is  defined  by 
assigning  a level  of  the  parameter  which  it  is  desired  to  accept  with 
probability  1-a,  giving  (Rs>  1-a)  in  Figure  1 (page  9).  The  other 
point  is  selected  for  an  undesirable  level  of  the  parameter  for  which 
it  is  desired  to  have  a low  probability  (p)  of  acceptance.  Figure  1 
(page  9)  illustrates  tiie  fact  that  even  though  all  the  tests  repre- 
sented by  the  O.C.  curves  pass  through  the  point  (Rs,  1-a),  there  is  a 
large  difference  in  their  ability  to  reject  values  of  R close  to  R$. 

Building  on  these  results,  the  following  chapter  presents  a 
literature  review  of  reliability  testing  procedures,  along  with  their 
underlying  assumptions,  and  describes  their  methods  of  application. 


CHAPTER  III 

LITERATURE  REVIEW  OF  RELIABILITY  TESTING  PROCEDURES 

As  discussed  previously,  reliability  testing  plans  are  actually 
tests  of  hypothesis  in  which  decision  criterion  are  tabulated  for  a 
variety  of  test  parameters.  Underlying  any  such  set  of  testing  plans 
is  a model  which  completely  describes  the  conditions  under  which  the 
tests  are  valid.  Assumptions  included  in  the  test  are  the  distribu- 
tion of  the  random  variable  in  the  test,  the  method  of  test  sampling, 
and  sometimes  limiting  assumptions  which  allow  for  the  use  of  approxi- 
mations in  designing  the  test.  For  example,  a typical  test  may  assume 
that  the  random  variable  time  to  failure  is  exponentially  distributed, 
that  samples  are  taken  randomly  from  the  population  being  considered, 
and  that  the  random  variable  under  consideration  is  homogeneous 
throughout  the  population.  In  this  chapter  various  testing  plans  are 
discussed,  and  their  underlying  assumptions  are  noted.  Any  misappli- 
cation of  these  plans  would  result  in  invalid  results. 

Some  models  may  complicate  the  derivation  of  the  mathematical 
relationships  to  the  degree  that  esoteric  mathematical  methods  of 
approximations  must  be  used.  However,  any  test  presented  in  this 
chapter  can  be  represented  by  a standard  Operating  Characteristic 
curve,  which,  in  many  cases,  is  published  is  a part  of  the  testing 
procedure. 

AGREE  Task  Group  III  Procedure 

The  Advisory  Group  on  Reliability  of  Electronic  Equipment  (AGREE) 
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was  established  by  the  Department  of  Defense  in  1952.  In  1957  this 
group  published  a report  which  contained  procedures  for  testing  for 
reliability,  (17).  These  procedures  were  designed  for  testing  compo- 
nents which  could  be  repaired  or  replaced  in  service  after  failure  and 
can  be  only  partially  applied  to  the  problem  of  testing  for  the  reli- 
ability of  parts  which  are  normally  disposed  of  after  failure.  Since 
their  publication,  these  procedures  have  been  included  in  many  con- 
tracts for  military  electronic  equipment. 

The  procedure  uses  a sequential  method  for  testing  the  hypothesis 
that  the  true  mean  time  to  failure  e of  the  items  under  test  is  greater 
than  a specified  value  ej.  The  tests  are  based  on  the  work  of  Epstein 
and  Sohel  (10),  in  which  n items  are  placed  on  test,  with  a failing 
item  either  replaced  or  repaired.  The  test  is  continued  until  a deci- 
sion can  be  made  by  the  sequential  decision  rules.  The  test  assumes 
that  the  times  between  failures  for  each  item  on  test  is  from  an  ex- 
ponential distribution  with  parameter  e.  The  test  procedure  is  com- 
pletely specified  by  the  selection  of  the  contract  specification  of 
time  to  failure  elt  The  operating  characteristics  of  the  test  are 
such  that  a,  the  probability  of  a reject  decision  when  9=6i,  is  .10, 
and  8,  the  probability  of  an  accept  decision  for  6=2/30!,  is  also  .10. 
The  test  results  are  independent  of  the  number  of  items  n put  on  test 
originally,  so  that  the  procedure  provides  a means  of  determining  n 
so  as  to  minimize  the  testing  time  needed  to  reach  a decision  given 
the  constraints  of  production  rates  and  the  specified  value  of  ej. 

The  procedure  also  provides  a means  of  relating  the  results  of 


the  test  to  other  untested  items  using  an  ^'.tributes  sampling  plan. 
Having  identified  each  item  tested  as  either  satisfactory  or  unsatis- 
factory a?  far  as  reliability  is  concerned,  a multi-level  attribute 
sampling  plan  is  applied  to  make  decisions  pertaining  to  the  remainder 
of  the  month's  production  which  has  not  bcun  tested.  If  an  accept 
decision  is  obtained  from  the  multi-level  attribute  sampling  plan, 
the  remainder  of  the  month's  production  is  accepted  without  testing. 
Otherwise,  every  item  will  be  tested  prior  to  acceptance.  The  rules 
of  the  multi-level  attributes  sampling  plan  are  as  follows: 

(1)  If  the  first  22  items  are  found  to  be  satisfactory,  the 
remainder  of  the  month's  production  is  accepted  without  further  test- 
ing. 

(2)  If  a single  unsatisfactory  item  is  discovered,  the  next  37 
items  must  be  found  satisfactory  in  order  to  accept  the  remainder  of 
the  month's  production  without  further  testing. 

(3)  If  more  than  one  unsatisfactory  item  is  discovered,  the  next 
50  items  must  be  found  satisfactory  in  order  for  the  remainder  of  the 
month's  production  to  be  accepted  without  testing. 

Once  a process  qualifies  for  acceptance  without  testing,  the 
items  produced  by  that  process  during  the  remainder  of  the  month  are 
accepted  without  testing.  At  the  beginning  of  the  following  month, 
however,  a small  number  of  items  must  be  tested.  The  exact  number  is 
not  specified,  but  the  document  indicates  that  the  number  would  be  on 
the  order  of  from  four  to  ten  items.  If  no  unsatisfactory  items  are 
detected,  all  items  will  be  tested  according  to  the  original 
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multi-level  attributes  sampling  rules  until  the  process  again  quali- 
fies for  acceptance  without  testing. 

This  procedure  is  not  without  criticisms.  As  noted  by  Dellin- 
ger (5),  although  the  method  refers  to  itself  as  a continuous  attri- 
butes sampling  plan,  it  does  not  provide  a known  level  of  protection 
as  do  most  continuous  sampling  plans.  Since  the  procedure  does  not 
require  the  continuous  selection  of  items  to  be  tested  on  a proba- 
bilistic basis  to  detect  changes  in  quality  level,  overall  risk  cannot 
be  determined  statistically.  The  procedure  is  also  designed  only  for 
use  on  a regular  production  process  basis,  and  is  clearly  not  appli- 
cable to  the  testing  of  pre-production  prototype  items.  However,  the 
basics  by  Epstein  and  Sobel,  on  which  the  procedure  was  developed, 
does  contain  ideas  which  can  be  applicable,  as  discussed  later. 

H-108  Procedures 

Quality  Control  and  Reliability  Handbook  H-108,  Sampling  Proce- 
dures and  Tables  for  Life  and  Reliability  Testing  (18),  was  published 
by  the  Office  of  the  Assistant  Secretary  of  Defense  in  1960  to  provide 
standard  sampling  plans  for  life  and  reliability  testing  relative  to 
established  reliability  requirements  in  government  contracts.  It  has 
been  distributed  widely  among  government  agencies  and  contractors,  and 
is  designed  to  be  used  as  a standard  reference  by  reliability  and 
quality  control  engineers.  The  plans  are  derived  from  the  results  of 
Epstein  and  Sobel  (10)  and  Epstein  (8). 

All  of  the  plans  are  based  on  the  assumption  of  an  underlying 
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exponential  distribution  of  time  to  failure 

f(t)  = \ e“t/e  (3.1) 

where  e is  the  mean  time  to  failure.  A number  of  items  n are  randomly 
selected  and  placed  on  test,  and  observations  are  made  of  times  to 
failure  or  the  number  of  failures  in  a given  time.  It  is  assumed  that 
the  lot  is  homogeneous;  that  is,  that  all  items  in  the  lot  have  the 
same  distribution  of  time  to  failure.  As  a result  of  the  sampling 
plan  a decision  is  made  whether  the  lot  meets  specified  reliability 
requirements.  The  plans  presented  are  applicable  to  testing  with  and 
without  replacement.  For  testing  with  replacement,  failed  items  are 
either  replaced  or  repaired,  and  the  plan  assumes  that  times  between 
failures  can  be  approximated  by  an  exponential  distribution  with 
mean  e. 

Three  different  types  of  procedures  are  presented  in  this  hand- 
book. The  first  type  is  made  up  of  tests  which  are  terminated  upon 
occurrence  of  a preassigned  number  of  failures.  The  second  type  con- 
sists of  tests  which  are  terminated  at  preassigned  times,  and  the  third 
type  is  a group  of  sequential  test  plans.  An  important  feature  of  both 
the  first  and  third  types  is  that  the  operating  characteristics  of  the 
plans  are  independent  of  the  number  of  items  placed  on  test.  Conse- 
quently, for  any  particular  O.C.  curve  there  are  a large  number  of 
plans  each  with  its  own  sample  size.  The  handbook  provides  a cost 
model  which  allows  determination  of  the  optimal  sample  size  by  balanc- 
ing the  cost  of  placing  items  on  test  with  the  cost  of  waiting  for  the 
decision  resulting  from  the  test.  The  selection  of  sample  size  does 


not  determine  the  amount  of  testing,  only  how  the  testing  will  be 
spread  over  time. 
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The  plans  of  the  second  type  mentioned  above,  where  testing  is 
terminated  at  a predetermined  time,  may  be  terminated  earlier  if  a 
specified  number  of  failures  occurs.  For  these  plans  the  selection 
of  the  O.C.  curve  and  the  termination  time  completely  specify  the 
plan,  including  sample  size.  This  type  plan  is  useful  when  practical 
considerations  require  that  a test  cannot  exceed  a set  deadline. 

All  of  the  plans  described  in  this  handbook,  along  with  their 
underlying  theory,  can  be  applied,  with  some  modification,  to  the 
testing  of  prototype  items.  The  greatest  limitation  of  these  plans 
is  the  assumption  of  an  exponential  distribution  of  times  to  failure. 
Although  Oavis  (4),  et  al.,  has  given  evidence  that  many  failures  do 
occur  randomly  in  time,  and  it  has  been  assumed  largely  that  failures 
will  follow  such  a distribution,  there  is  evidence  that  such  an  assump- 
tion is  many  times  invalid.  In  the  case  of  prototype  small  caliber 
arms,  the  assumption  would  be  that  the  number  of  rounds  between  fail- 
ures can  be  approximated  by  an  exponential  distribution.  Further  dis- 
cussion of  this  follows  in  later  chapters.  In  any  case,  any  testing 
plan  except  for  non-parametric  tests  can  be  subject  to  the  criticism 
of  assuming  a distribution  which  may  rot  be  valid. 

TR3  Procedures 


Quality  Control  and  Reliability  Technical  Report  TR3,  Sampling 
Procedures  and  Tables  for  Life  and  Reliability  Testing  Based  on  the 
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Wei bull  Distribution,  (12),  was  published  by  the  Department  of  De- 
fense in  September,  1961.  Since  the  exponential  distribution  is  but 
a special  case  of  the  Wei bull  distribution,  this  report  can  be  con- 
sidered a generalization  of  one  form  of  the  testing  procedure  present- 
ed for  the  exponential  case  in  H-108.  The  testing  plans  are  for  the 
case  where  n items  are  placed  on  test  and  the  test  terminated  when 
r(r<n)  failures  occur*  or  at  time  Tj,  whichever  occurs  first.  The 
plans  provide  only  for  the  caso  of  non-replacement  of  failed  items, 
and  the  underlying  theory  is  further  described  by  Goode  and  Kao  (11). 

The  criterion  for  the  test  is  the  mean  time  to  failure.  A three  para- 
meter model  is  used,  however  it  is  assumed  that  two  of  the  parameters, 
y and  3,  are  known.  The  density  function  of  the  three  parameter  Wei- 
bull  distribution  is  given  by 

f(t;Y,M)  - (s/n)  <3-2) 

where 

Y = location  parameter, 

3 = shape  parameter,  and 

n = scale  parameter. 

The  plans  in  TR3  are  based  on  the  assumption  thatY-0;  however, 
procedures  are  included  for  making  a simple  adjustment  in  the  test  I 

| plan  if  y is  not  zero. 

't 

[ The  test  procedure  given  in  TR3  is  as  follows: 

(1)  Select  at  random  a sample  of  n items  from  a lot  size  of  N. 

(2)  Place  the  sample  items  on  life  test  for  some  preassigned 


period  of  time  T. 
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(3)  Denote  by  y the  number  of  failures  observed  prior  to  time  T. 

(4)  Accept  the  lot  if  y<C,  where  C is  an  acceptance  number  speci- 
fied by  the  plan,  but  if  y>C,  reject  the  lot. 

O.C.  curves  are  not  provided  in  TR3  for  the  available  plan.  How- 
ever, tables  are  provided  which  give  values  of  C and  the  minimum  sam- 
ple size  n for  various  levels  of  protection.  Also  included  are  graphi- 
cal procedures  for  estimating  the  parameters  of  a Wei  bull  distribution, 
as  well  as  determining  if  a given  set  of  data  came  from  a Wei bull  dis- 
tribution. 

The  advantage  of  these  plans  are  that  they  provide  a reliability 
testing  procedure  which  allows  for  a more  general  assumption  of  the 
underlying  distribution  of  the  parameter  of  interest.  However,  the 
plans  are  limited  to  testing  without  replacement  only,  which  limits 
their  efficiency. 


TR4  Procedures 

Quality  Control  and  Reliability  Technical  Report  TR4  Sampling 
Procedures  and  Tables  for  Life  and  Reliability  Testing  Based  on  the 
Weibull  Distribution  (13)  was  developed  to  supplement  the  plans  pre- 
sented in  TR3.  The  only  significant  difference  between  the  test 
plans  in  TR3  and  those  in  TR4  lies  in  the  parameter  used  to  define 
reliability.  In  TR3,  the  mean  life  (mean  time  to  failure)  was  used. 

In  TR4,  the  hazard  (failure)  rate  z(t)  is  used. 

In  both  TR3  and  TR4  the  plans  are  attribute  sampling  plans  based 
on  the  probability  p of  a single  item  failing  prior  to  the  test  termi- 
nation time  T.  In  TR4  the  relationship  between  the  hazard  rate  z(t) 


and  p is  established  as  follows.  The  hazard  rate  is  defined  by 


z(t)  = f (t)/R(t) 

(3.3) 

where,  in  the  case  of  the  Wei bull  distribution, 
f(t)  = B/n  (t/n)e“1e”(t/n)  B,n,t>0 

(3.4) 

and 

R(t)  = 1 - F(t)  = e"^t/n^ 

(3.5) 

giving 

z(t)  = (B/n)  (:;/n)B-1. 

(3.6) 

Multiplying  by  (t/e)  gives 
(tz(t)/f$)  = (t/n)6 

(3.7) 

and  it  is  seen  that 

p = F{ t)  = 1 - R(t)  = 1 - e"^t/n^ 

(3.8) 

which,  upon  substitution,  gives 
p = ! . e-tz(t)/6. 

(3.9) 

This  gives  the  desired  relationship  between  z(t)  and  the  proba- 
bility p of  a single  item  failing  prior  to  time  t.  This  relationship 
can  be  used  to  obtain  the  value  of  p to  use  in  the  attributes  plan  by 
simply  setting  t = T,  the  test  termination  time. 

The  testing  plans  in  TR4  are  of  the  same  general  form  as  in  TR3. 
There  is  one  difference,  however,  in  that  it  is  assumed  by  all  plans 
in  TR4  that  the  test  time  will  be  the  same  as  the  value  of  t for  which 
z(t)  is  specified.  When  this  is  not  the  case,  a table  of  conversion 
factors  is  provided  which  allows  the  conversion  of  the  specified  ha- 
zard rata  for  time  t into  an  equivalent  hazard  rate  for  the  desired 
test  time.  In  addition  to  the  plans  based  on  a specified  hazard  rate 


20 

the  report  also  contains  instructions  for  sampling  when  the  average 
hazard  m(t)  is  specified.  The  average  hazard  is  defined  as  the  cumu- 
lative hazard  M(t)  divided  by  t.  That  is, 

m(t)  = M(t)/t  = (1/t)  /Vy)  dy.  (3.10) 

— oo 

In  the  appendix  to  TR4  it  is  shown  that  for  any  suitable  hazard 
function  z(t), 

R(t)  - e - / z<x>  dx.  (3.11) 

M 00 

Therefore,  if  m(t)  is  specified,  p can  be  determined  without  determin- 
ing the  form  of  z(t), 

p = 1 - e"tm(t).  (3.12) 

From  this  relationship  nonparametric  tests  can  be  devised  using  any 
suitable  attributes  sampling  plan  with  p as  the  percent  defective. 

In  practice,  as  noted  by  Dellinger  (5),  this  form  of  test  can  be 
very  desirable  since  no  assumption  need  be  made  concerning  the  distri- 
bution of  time  to  failure.  Specifying  reliability  in  terms  of  percent 
failure  for  some  given  time  period  would  allow  use  of  this  plan  as  an 
attributes  sampling  plan  to  obtain  a nonparametric  reliability  test. 
However,  to  accomplish  this,  the  test  time  must  equal  the  value  of  t 
for  which  m(t)  is  specified.  If  one  desires  to  test  for  a period  of 
time  other  than  the  time  for  which  m(t)-  is  specified  it  is  necessary 
to  use  conversion  tables  in  TR4  which  results  in  an  assumption  of  a 
Wei  bull  distribution  and  a value  for  the  parameter  3.  Also,  if  the 
time  t for  which  m(t)  is  specified  is  other  than  the  required  time  of 
operation  t'  for  which  R(t')  is  desired,  the  test  results  can  be 
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applied  to  the  calculation  of  R(t*)  only  after  a form  for  the  distri- 
bution of  time  to  failure  has  been  assumed.  In  other  words,  one  can 
utilize  a truly  nonparametric  test  only  if  the  time  t for  which  m(t) 
is  specified  is  equal  to  both  the  test  time  and  the  time  t'  for  which 
R(t')  is  desired. 

It  is  doubtful  that  reliability  would  be  specified  in  terms  of 
m(t)  when  there  is  knowledge  of  the  form  of  the  distribution  of  time 
to  failure.  It  would  be  more  logical  to  specify  reliability  in  terms 
of  the  unknown  parameter  of  the  distribution.  Similarly,  there  is 
little  reason  to  specify  reliability  in  terms  of  the  hazard  rate 
(z(t) ) if  the  distribution  of  times  to  failure  is  known.  It  is  diffi- 
cult to  visualize  any  practical  situation  where  the  specification  of 
reliability  in  terms  of  the  hazard  rate  or  average  hazard  would  make 
much  sense.  Such  specifications  only  confuse  the  basic  issue  of 
establishing  the  reliability  of  an  item.  For  these  reasons,  in  addi- 
tion to  those  given  for  the  TR3  report,  this  method  shows  little  pro- 
mise for  the  problem  under  consideration  by  this  report. 

Procedures  Based  on  the  Gamma  Distribution 

Gupta  and  Groll  (14)  discuss  sampling  plans  which  are  a generali- 
zation to  the  gamma  distribution  of  the  truncated  life  tests  without 
replacement  for  the  exponential  distribution  in  H-108.  It  is  similar 
to  the  generalization  of  the  same  plans  to  the  Wei  bull  distribution  in 
TR3  and  TR4.  A relationship  is  developed  between  the  single  unknown 
parameter  of  the  distribution  and  the  probability  p of  a single  item 
failing  prior  to  the  end  of  the  test  termination  time.  Standard 


attribute  sampling  procedures  are  then  applied  using  p as  the  para- 
meter of  a binomial  distribution.  Testing  procedures  are  identical 
to  those  in  TR3. 

It  is  assumed  in  these  plans  that  times  to  failure  are  approxi- 
mated as  a random  variable  with  a gamma  distribution  density  function 
f(t)  = l/v(a)  {x/e}0”^76  for  t,a,e>0.  (3.13) 

The  cumulative  distribution  function  is  then 

F(t)  =QJt  l/v(ct)  {x/e}a_1e"x/0  dx» 

and  the  reliability  function  thus  is 

R(t)  = 1 - F(t).  (3.15) 

It  is  assumed  that  the  parameter  a is  known,  so  that  the  probabi- 
lity p of  a single  item  failing  prior  to  time  t,  in  terms  of  t and  the 
unknown  parameter  e,  is 

p = F(t,e)  • (l/v(a))  /t(x/e.)a'1e‘(x/e)  dx.  (3.16) 

0 

The  mean  of  the  gamma  distribution,  E(t)  = ae,  is  used  as  the 

acceptance  criterion  in  these  plans.  The  sampling  plans  are  indexed 

by  the  ratio  t/a0Q,  in  which  t is  equal  to  the  test  termination  time, 

a is  equal  to  the  known  shape  parameter  of  the  distribution,  and  aeQ 

is  a specified  value  of  the  mean  time  to  failure  ae.  The  specified 

value  of  ae  is  a minimum  value  for  which  it  is  desired  to  reject  the 
o 

lot  with  a high  probability,  which  places  emphasis  on  the  consumer's 
risk  rather  than  the  more  common  practice  of  emphasizing  the  producer's 
risk. 


Several  methods  of  selecting  sampling  plans  are  available, 


23 

including  the  method  of  choosing  an  0,C.  curve  and  finding  a matching 
plan.  However,  only  a limited  number  of  OX.  curves  are  presented. 

The  suggested  method  is  as  follows: 

(1)  Determine  the  value  of  the  ratio  t/ct©o  and  for  p*  such  that 
P (Rejecting  the  lot  when  ae=a6o)  = p*.  The  time  to  terminate  the  test 
is  arbitrary,  and  selecting  a0Q  and  p*  results  in  choosing  a point  on 
the  O.C.  curve  of  the  test  corresponding  to  the  consumer's  risk. 

(2)  A table  is  provided  which  gives  a family  of  plans  correspond- 
ing to  the  ratio  t/a0o  and  p*.  Each  plan  is  specified  by  the  numbers 

C is  equal  to  the  acceptance  number  and  n is  equal  to  the  minimum  sam- 
pTe  size. 

(3)  A value  for  a0  is  choosen  for  which  it  is  desired  to  accept 
the  lot  with  a probability  of  .95.  Tabulated  values  are  given  for 
acceptance  numbers  C which  have  this  desired  property.  The  desired 
plan  will  then  be  the  number  of  the  family  of  plans  selected  in  step  2 
having  the  acceptance  number  found  in  step  3. 

A rather  broad  range  of  plans  is  presented,  with  procedures  in- 
cluded for  the  selection  of  plans  based  on  their  operating  character- 
istics. The  report  doesn't  give  any  guidance  relative  to  the  selec- 
tion of  an  economic  test  termination  time  which  minimizes  test  costs, 
as  given  in  H-108.  Neither  is  it  indicated  how  to  determine  whether 
a set  of  failure  data  can  be  approximated  by  a gamma  distribution. 
However,  the  procedures  are  useful  in  that  they  specify  how  to  test 
for  reliability  when  it  is  assumed  that  failure  times  are  approximated 
by  a rather  general  density  function,  the  gamma.  However,  the  tests 


24 


are  again  limited  to  testing  without  replacement. 

Attributes  Sampling  Plans 

In  the  area  of  quality  control  there  are  many  attribute  sampling 
plans  which  can  be  applied  to  the  problem  of  reliability  testing.  It 
is  necessary  only  that  the  test  be  conducted  in  which  success  or  fail- 
ure are  possible  outcomes;  for  example,  when  there  is  nc  required  time 
of  operation,  such  a test  would  be  the  operation  of  the  device  in  the 
specified  environment.  In  those  cases  where  the  definition  of  success 
requires  operation  for  a specified  time,  an  attribute  test  would  be 
one  in  which  the  device  is  tested  for  the  specified  operation  time  and 
each  trial  judged  success  or  failure  depending  on  whether  the  device 
operated  properly  for  the  test  period.  Such  tests  are  usually  called 
nonparametric  or  distribution- free  tests. 

Where  such  tests  can  be  applied,  it  is  assumed  that  each  device 
in  the  lot  has  the  same  probability  of  failure  p and  that  reliability 
is  simply  R = 1 - p.  The  actual  distribution  of  the  number  of  failures 
x in  a sample  size  n drawn  from  a lot  size  N is  known  to  be  hypergeo- 
metric, Duncan  (6).  However,  when  n is  small  compared  to  N,  as  is 
usually  the  case,  a binomial  distribution  can  be  assumed  with  little 
error.  Most  attribute  sampling  plans  are  based  on  the  binomial  dis- 
tribution. 

The  principal  advantage  of  attribute  sampling  plans  lies  in  the 
fact  that  one  does  not  need  to  know  anything  about  the  underlying 
distribution  of  time  to  failure.  The  disadvantage  is  that  more  time 
is  required  to  make  a decision  using  an  attributes  plan  than  if  one 
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used  a plan  based  on  a known  distribution  of  time  to  failure  with 
equivalent  operating  characteristics. 

Summary 

The  documents  discussed  in  this  chapter  are  representative  of  the 
standard  reliability  testing  techniques  generally  available  today. 

A sample  of  other  books  and  articles  on  the  subject  failed  to  present 
any  techniques  which  are  basically  different  from  those  considered  in 
this  chapter,  although  many  minor  variations  of  the  techniques  dis- 
cussed can  be  found. 

The  general  form  of  all  plans  were  similar,  patterned  after  stan- 
dard statistical  quality  control  plans  developed  in  the  last  decade. 

The  emphasis  is  on  the  mechanism  of  application,  while  basic  assump- 
tions are  somewhat  neglected  and  the  statistical  basis  for  each 
usually  relegated  to  footnotes.  However,  they  are  easily  applicable 
in  many  practical  situations. 

None  of  the  plans  are  directly  applicable  to  the  problem  of 
testing  prototype  small -caliber  arms  for  reliability.  All  assume  the 
random  variable  of  interest  is  time  to  failure,  and  many  assume  a 
regular  production  process  is  ongoing.  Except  for  the  plans  given  in 
H-108,  which  will  be  further  discussed  in  Chapter  V,  all  plans  are  for 
tests  without  replacement,  which  makes  them  relatively  useless  for  arms 
testing.  The  following  chapter  describes  the  testing  procedure  pre- 
sently used  by  the  Army  to  test  prototype  arms. 


CHAPTER  IV 


t 

I- 

EVALUATION  OF  PRESENT  ARMY  PROTOTYPE  TESTING  METHODS 

This  chapter  presents  a general  discussion  of  the  procedures 

s 

| presently  used  by  the  U.S.  Army  for  testing  prototype  small  caliber 

arms.  Also  discussed  are  the  deficiencies  of  the  present  procedures. 

I 

In  the  early  development  of  a small  caliber  weapon  the  Army  re- 

j 

ceives  sealed  bids  for  development  of  a specific  number  of  prototypes 
from  private  contractors.  Of  these  bidders,  as  many  as  four  or  five 
are  awarded  contracts  to  develop  prototypes.  When  delivered  by  the 
contractors  these  weapons  undergo  engineering  design  tests,  which  in- 
clude testing  for  reliability,  one  of  several  factors  used  to  deter- 

,,  mine  which  contractor  will  continue  development  of  the  weapon.  Occa- 

sionally, two  contractors  continue  developing  their  prototype  models 
because  each  has  certain  desirable  attributes  not  found  in  the  other. 

Experience  has  shown  that  three  failure  modes  occur  in  the  wea- 
ponry, Brown  (1): 

(1)  part. breakage  - the  actual  breakage  of  a part  of  the  gun; 

(2)  non-immediately  clearable  - gun  jam  which  takes  some  period 
of  time  to  clear; 

(3)  immediately  clearable  - gun  jam  which  can  be  cleared  imme- 
diately. 

When  any  assumptions  at  all  are  made  about  the  basic  underlying 
distribution  of  the  number  of  rounds  between  failures,  it  has  been 
assumed  that  each  failure  mode  can  be  approximated  by  independent 
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exponential  distributions,  with  each  having  its  own  mean  number  of 
rounds  to  failure  e.  The  use  of  the  exponential  distribution  to 
approximate  the  number  of  rounds  to  failure  for  each  type  of  failure 
assumes  that  each  type  of  failure  occurs  independently  only  when  an 
infrequent  random  environmental  excursion,  greater  than  some  magnitude 
Ec,  occurs  in  the  environment  E in  which  the  weapon  is  operating.  It 
is  difficult  to  determine  what,  if  any,  data  analysis  had  been  per- 
formed to  justify  the  assumption  of  the  exponential  distribution  for 
each  failure  mode. 

The  Army  contract  usually  contains  a reliability  requirement 
which  specifies  a minimum  number  of  rounds  between  failures  for  each 
failure  mode,  although  there  is  quite  a bit  of  variation  in  the  form 
of  the  specifications.  The  following  example  from  the  Light  Weight 
Individual  Weapon  Systems,  Final  Report  (2)  illustrates  one  form  used: 

"These  requirements  establish  reliability  limits  as  follows: 

(1)  No  more  than  two  stoppages  clearable  by  immediate  action  in 
1000  rounds  (2000  rounds  desirable) 

(2)  No  more  than  one  stoppage  not  clearable  by  immediate  action, 
but  not  requiring  parts  replacement  in  6000  rounds  (10,000  rounds  de- 
sirable) 

(3)  Not  more  than  two  malfunctions  requiring  parts  replacement 
by  operator  in  6000  rounds  (10,000  rounds  desirable)." 

The  exact  form  of  the  contract  requirement  for  reliability  may 
vary  rather  extensively  from  contract  to  contract  and  between  different 
weapon  programs,  but  these  basic  malfunction  modes  are  usually  given 
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a required  number  of  rounds  between  failures. 

It  is  important  to  note  that  all  of  these  specifications  stipu- 
late only  a point  estimate  requirement  and  ignore  that  the  number 
of  rounds  between  failures  is  a random  variable  which  may  vary  quite 
a bit  in  a manner  described  by  the  form  of  its  actual  underlying  pro- 
bability density  function.  Values  for  the  a and  b levels  are  not 
specified,  and  in  fact  the  entire  statistical  nature  of  the  random 
variable  appears  to  have  been  ignored.  A requirement  that  "10,000 
rounds  is  desirable"  is  clearly  of  little  use  - as  discussed  previous- 
ly, only  a lower  limit  is  usually  specified  practically,  and  any  num- 
ber exceeding  this  value  is  desirable. 

Project  SAW  (Squad  Automatic  Weapon),  which  is  underway  at  the 
time  of  this  writing,  provides  an  example  of  an  on-going  Army  project 
in  this  area.  Project  SAW  is  the  development  by  ARMCOM  (U.S.  Army 
Armament  Command)  of  a 6 millimeter  light  machine  gun,  weighing  less 
than  20  pounds,  with  a 200  round  clip,  capable  of  firing  500  rounds,  per 
minute,  fully  automatic.  The  present  contract  calls  for  each  of  3 con- 
tractors to  produce  12  guns.  Forty  thousand  experimental  rounds  are 
available  at  a purchase  cost  of  $1  per  round,  with  TEC0M  (U.S.  Army 
Test  and  Evaluation  Command)  performing  the  actual  testing  at  an  addi- 
tional cost  to  ARMCOM.  Testing  costs  are  determined  for  each  indivi- 
dual weapon  system.  For  small  arms  it  usually  ranges  around  $50,000 
for  the  use  of  ranges,  personnel  and  other  overhead.  As  mentioned, 
the  experimental  ammunition  costs  $1  per  round  presently;  this  will  be 
reduced  about  50%  in  one  year  as  the  production  process  becomes  more 
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efficient,  and  eventually  will  range  around  $.10-$. 20  per  round  as 
full  production  commences.  The  present  contract  dictates  delivery 
of  10  new  usable  weapons  and  two  weapons  which  undergo  a contractor 
demonstration  test.  Table  1 (page  30)  gives  the  actual  cost  data 
associated  with  this  stage  of  the  program. 

Table  2 (page  31)  gives  a summary  of  weapon  and  rounds  usage 
during  a typical  engineering  design  test  - in  this  case,  from  tests 
of  the  XM207E2  machine  gun,  Stoner  EDT  (19).  As  indicated,  out  of  10 
weapons  only  two  are  used  for  testing  for  reliability  during  the 
accuracy  and  endurance  test,  firing  a total  of  72,667  rounds.  The 
table  indicates  the  various  tests  the  armament  is  subjected  to  in 


addition  to  the  reliability  test.  Many  of  the  tests,  such  as  the  mud 
test,  dust  test  or  icing  test  are  controlled  condition  tests  where  the 
operation  of  the  weapon  is  tested  under  the  extreme  conditions  it  may 
be  subjected  to.  Other  tests  are  accuracy  tests,  and  a third  group  of 
tests  are  such  tests  as  sound  level,  smoke,  and  flash  tests  where  it 
is  desirable  to  minimize  such  environmental  conditions  resulting  from 
firing  the  weapon.  All  of  the  tests  are  designed  to  check  if  the  arms 
under  test  meet  specified  criteria.  For  example,  in  the  cookoff  test 
it  may  be  desired  to  determine  the  number  of  rounds  fired  continuously 
from  a cool  gun  which  will  cause  a chambered  round  to  "cook  off"  with- 
in 30  minutes.  The  requirement  may  be  that  the  weapon  will  not  cook 
off  after  firing  less  rounds  than  the  determined  critical  cook  off 
level  (assuming  stoppage  on  a chambered  round). 

Table  3 presents  a summary  of  weapon  and  rounds  usage  during 
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Mareraont 

Philco-Ford 

Rodman 

Base  Line 

$427K 

$460K 

# 

First  Overrun 

$51K 

*108K 

# 

Extension 

S99K 

$98K 

K 

Magazine 

$13K 

$17K 

# 

Second  Overrun 

8 OK? 

$68K 

* 

Total  . 

*590K 

I751K 

* 

* No  actual  contract  exists  on  Rodman  ( Rock 
Island  Developed  Fixture  ) but  it  will  probably 
equal  if  not  exceed  the  figures  given. 


Table  1 - Project  SAW  Cost  Information 


Table  2 - XM207E2  EDT  Weapon  and  Hounds  Usage 


Table  3 - Stoner  EDT  Weapon  and  Rounds  Usage  . 
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another  engineering  design  test  on  another  weapon  system,  from  the 
Stoner  EOT  (19).  Comparison  of  Table  2 (page  31)  with  Table  3 (page 
32)  demonstrates  the  wise  disparity  between  tests  of  similar  systems. 

In  this  set  of  tests  15  weapons  are  used,  with  five  weapons  used  for 
reliability  and  accuracy  testing,  with  a total  of  121,967  rounds  fired. 
In  V m tests  a maximum  mean  life  of  36,000  rounds  between  failures 
was  being  tested.  The  variation  between  the  tests  summarized  in 
Table  2 and  those  in  Table  3 is  found  in  all  engineering  design  tests, 
as  not  only  is  there  no  set  procedures  for  these  tests,  but  the  objec- 
tives of  the  tests  vary  among  weapon  systems  due  to  the  special  re- 
quirements of  different  weapons. 

The  reliability  tests  used  in  these  tests  are  of  a very  arbitrary 
nature.  The  number  of  both  guns  and  rounds  fired  are  not  selected  on 
a theoretical  basis,  but  rather  on  a "rule  of  thumb"  that  a few  wea- 
pons firing  close  to  the  maximum  number  of  rounds  mentioned  in  the 
specification  should  indicate  whether  or  not  the  reliability  of  the 
weapon  meets  contracted  values.  The  simple  arithmetic  average  of  the 
number  of  rounds  between  failures  is  used  as  the  estimate  of  the  para- 
meter e,  mean  rounds  between  failure,  for  each  failure  mode,  with  no 
consideration  given  to  the  distribution  of  the  random  variable. 

As  will  be  shown,  ignoring  the  distribution  of  e in  these  tests 
results  in  a much  less  efficient  test,  and  may  increase  3,  the  proba- 
bility of  deciding  that  e is  greater  than  or  equal  to  the  contracted 
value  when  is  actually  less  than  the  specified  value.  Such  an  erron- 
eous decision  would  greatly  increase  the  overall  life  cycle  cost  of 
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CHAPTER  V 

TRUNCATED  AND  SEQUENTIAL  LIFE  TESTING 

This  chapter  further  considers  two  testing  methods  mentioned 
previously  in  terms  applicable  to  testing  of  prototype  small  caliber 
arms.  Throughout  the  chapter  it  is  assumed  that  the  number  of  rounds 
between  failures  for  each  failure  mode  follows  an  underlying  exponen- 
tial probability  density  function,  with  each  failure  mode  having  its 
own  mean  number  of  rounds  between  failures  e. 

All  of  the  testing  procedures  which  were  discussed  previously 
were  concerned  with  testing  whether  the  mean  time  between  failures 
met  or  exceeded  a specified  value.  In  the  case  of  testing  small 
caliber  arms  the  test  is  not  concerned  with  times  between  failures, 
but  rather  tests  whether  the  mean  number  of  rounds  between  failures 
meets  or  exceeds  a specified  value.  The  procedures  for  the  two  types 
of  tests  are  similar,  however.  Testing  small -caliber  arms  can  be 
viewed  as  putting  n guns  on  test,  recording  the  number  of  rounds  to 
the  first  failure,  second  failure,  and  so  on,  while  repairing  each 
weapon  which  fails  (clear,  unjam,  or  replace  part,  depending  or.  which 
type  of  failure  occurs)  and  returning  the  gun  to  test.  This  is  analo- 
gous to  putting  n items  on  test,  and  testing  with  replacement,  as 
discussed  previously,  except  the  testing  is  independent  of  time  and 
concerned  rather  with  the  number  of  rounds  fired. 

This  chapter  presents  the  development  of  two  feasible  tests.  The 
first  test  discussed  is  a truncated  life  test,  where  the  test  is 
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terminated  either  on  the  rth  failure  or  on  a pre-assigned  number  of 
rounds  RQ.  The  other  test  is  a sequential  life  test  in  which  the  test 
continues  indefinitely  until  an  accept  or  reject  decision  is  reached 
on  the  basis  of  the  number  of  failures  within  a number  of  rounds  fired. 
Note  that  in  both  cases  the  term  "life"  refers  to  the  number  of  rounds 
to  failure  rather  than  the  number  of  time  units. 

Truncated  Life  Tests 

A truncated  life  test  proceeds  as  follows.  With  n guns  placed  on 
test,  it  is  decided  * ‘‘ance  that  the  experiment  will  be  terminated 
at  min(Xro,n,R0),  where  XrQ,n  is  a random  variable  equal  to  the  number 

1L 

of  rounds  fired  before  the  r0  failure  occurs,  and  RQ  is  the  trunca- 
tion number  of  rounds,  beyond  which  the  experiment  will  not  run.  Both 
rQ  and  RQ  are  assigned  before  experimentation  starts.  If  the  experi- 
ment is  terminated  at  XrQ>n  (that  is,  if  rQ  failures  occur  before  RQ 

rounds  are  fired),  then  the  action  in  terms  of  hypothesis  testing  is 
the  rejection  of  some  specified  null  hypothesis.  If  the  experiment  is 

1.L 

terminated  at.RQ  (that  is,  if  the  rQin  failure  does  not  occur  before 
round  number  RQ  is  fired),  then  the  action  in  terms  of  hypothesis 
testing  is  the  acceptance  of  some  specified  null  hypothesis. 

A previous  development  by  Epstein  (7)  is  applicable  here,  with  a 
simple  substitution  of  variable.  The  test  is  of  the  replacement  type 
where  the  test  is  started  with  n guns  and  any  gun  that  fails  is  re- 
placed or  repaired  tc  like  new  condition,  so  that  the  number  of  guns 
under  test  is  always  n.  Since  the  underlying  distribution  is 
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exponential  with  e equal  to  the  mean  number  of  rounds  between  failures, 
the  replacement  or  repair  of  failed  items  makes  the  life  test  a Poisson 
process  with  occurrence  rate  xe  = n/e. 

Thus,  the  probability  of  reaching  a decision  requiring  exactly  k 
failures  is 

P{r=k|e}  = p(k;x0Ro)  = (l/k‘.)e”nRo/e  (nR0/e)k, 


k 0,1,2,. o. ,r0-l, 


and 


l"n-l 


P{r=r0|e}  = 1-1  p(k;x0Ro). 


k=0 


(5.1) 

(5.2) 


The  expected  number  of  observations  to  reach  a decision  is 
ro 

E0(r)  = l kP(r=kje).  (5.3) 

k=0 

It  can  readily  be  shown  that  (5.3)  can  be  written  as 
r0-2  r0-l 

E0(r)  = x0Ro{  l p(k;X0Ro)}  + r0{l-  I p(k;xeR0)}.  (5.4) 

k=0  k=0 

Equation  (5.4)  is  a convenient  form  for  calculation.  For  any  pre- 
assigned n,  R0  and  r0,  E0(r)  can  be  found  easily  from  Molina's  tables 
of  the  Poisson  distribution  (16)  or  other  similar  tables. 

The  expected  number  of  rounds  to  reach  a decision  is  given  by 
E0(R)  = (e/n)E0(r).  (5.5) 


The  proof  of  (5.5)  is  as  follows.  It  can  be  shown  that 


>V1 


Ee(R)  - Ee(Xr ) + l p(k ;X6R0){R0  - Ee(xr  ,n|r=k)}  (5.6) 


k=0 


and 
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Ee(xr0»Jr=*c)  ~ + MXr0-k,n)»  k=l,2,...,r0“l.  (5.7) 

Furthermore, 

Ee(Xr0-k,n)  = Ee(Xro,n)  - E0(Xk,n)  * («o  - ««/"•  l<k<r0,  (5.8) 

since  the  unconditional  expected  number  of  rounds  fired  to  get  exactly 
s failures  in  a replacement  situation  is 

Ee(Xs,n)  * s0/n  (5.9) 

for  any  integer  s.  Substituting  (5.7)  and  (5.8)  into  (5.6)  gives 
(5.5). 

It  can  be  shown  that  (5.5)  can  be  rewritten  as 
Eo(R)  * I P(r=k|0)Ee(Xk  n)  (5.10) 

and  the  unconditional  expected  number  of  rounds  fired  is 

Ee(Xktn)  = k0/n*  (5.11) 

Suppose  the  truncation  rule  is  such  that  the  hypothesis  HQ: e=e0  is 
accepted  if  min(XrQ,njR0)  = R0,  that  is,  the  number  of  rounds  fired  re- 
quired to  observe  Xr  ,n  is  more  than  Rn.  Then  if  L(e)  is  defined  as 

o u 

the  probability  of  accepting  e=e0  when  e is  true,  it  follows  that 

L(o)  = 1 - P{r=r0 | e}  (5.12) 

where  P{r=r0|e}  is  given  by  Equation  (5.2). 

Let  X,  be  the  number  of  rounds  to  the  kth  failure  (whether  it 
k,n 

be  an  original  item  or  a replacement  item)  measured  from  the  beginning 
of  the  experiment.  It  has  been  shown  by  Epstein  and  Sobel  (9),  et  al., 
in  the  replacement  case,  that  if  one  starts  with  n items,  then  the 
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“best"  region  of  acceptance,  in  the  Neyman-Pearson  sense,  for  testing 

a hypothesis  H0  that  e=e0  against  alternatives  of  the  form  e^Ce^o), 

based  on  the  first  r number  of  rounds  to  failure  X,  w X,  . 0..,X  „ 

l,n,  2,n,  r,n, 

A 

is  of  the  form  a.,  >C,  where 
r,n 

®r,n  = nXr,n^r*  ^5*13^ 

It  follows  that  the  region  of  acceptance  for  Hq  is  of  the  form 
Xr  n>C*  = rC/na  Use  of  n>C*  as  a region  of  acceptance  means  in 

words  that  the  test  is  terminated  at  rninU^,  _,C*)  with  acceptance  of 

r *n 

Hq  if  truncation  occurs  at  Xf  n.  This  is  precisely  the  test  treated 

above  with  r=r„  and  C*  = R„. 

o o 

So  far  formulae  have  been  given  for  the  O.C.  curve,  expected  num- 
ber of  rounds  used  and  expected  number  of  failures  encountered  in  the 
course  of  reaching  a decision  for  any  preassigned  n,  R0,  and  rQ.  The 
following  is  a procedure  for  finding  the  appropriate  truncated  test 
(that  is,  for  finding  rQ  and  n)  when  the  truncation  number  of  rounds 
is  preassigned  and  the  O.C.  curve  is  required  (for  preassigned  type  I 
error,  a,  and  type  II  error,  e)  to  be  such  that  L(e0)>l-a  and  L(ei)<e. 
Here  eQ  and  Oj  are  preassigned  with  e0>e1. 

Epstein  and  Scbel  (9)  have  shown  that  the  best  acceptance  region 
of  size  a for  the  hypothesis  e=0o  (against  any  alternative  0^0^, 

based  on  the  first  r out  of  n failures,  for  preassigned  r and  n,  is 
®r,n>C  = 0oX2  (2r)/2r 


(5.14) 
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where  0 (2r)  is  a chi-square  variable  with  2r  degrees  of  freedom. 

In  order  that  the  test  have  an  O.C.  curve  for  which  L(e0)  = 1-a  and 

L(e1)£3,  we  need  to  choose  r suitably.  The  appropriate  values  of  r 

for  certain  values  of  a, 3,  and  0q/0 j are.  given  in  Table  4'.  For  values 

of  a, 3,  and  Qq/Sj  not  given  in  the  table,  the  appropriate  r to  use  is 

2 2 

the  smallest  integer  r such  that  xx_a  (2 r)/xp  Ur)^^. 

It  is  now  an  easy  matter,  in  the  replacement  case,  to  find  a 
truncated  test  meeting  the  conditions  prescribed  above.  The  appropri- 
ate rQ  is  given  by  the  values  in  Table  4,  and 

R0  = C*  = rC/n  = 0Qx^a  (2r)/2n.  (5.15) 

Since  n must  be  an  integer,  the  equality  can  be  satisfied  only  approxi- 
mately. For  all  practical  purposes  n can  be  chosen  as 

n = {eoXj-a(2r0)/2R0}  (5.16) 

where  {x>  means  the  greatest  integer  <x.  It  is  important  to  note  the 
appropriate  number  of  guns  on  test  n,  for  fixed  cc  and  3,  is  inversely 
proportional  to  the  truncated  number  of  rounds  RQ  fired  by  each  gun. 
Thus,  for  example,  to  reduce  the  truncation  number  of  rounds  fired  by 
each  weapon  on  test  by  a factor  of  2 requires  doubling  n,  the  number 
of  guns  on  test.  This  has  clear  implications  in  the  case  of  prototype 
testing,  where  a maximum  n is  set  by  the  number  of  prototypes  avail- 
able, requiring  enlarging  RQ  in  order  to  achieve  the  desired  a and  3 
levels.  It  is  clear  from  Equation  (5.16)  that  tabulated  values  of 
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xi-a  (2r0)/2  would  be  useful.  These  are  given  below  the  associated 
rQ  in  Table  4.  The  appendix  contains  alternate  tabular  forms  of  these 
results.  Appendix  A contains  tabulated  values  of  RQ/eo  and  r for 
values  of  e^e  ,«  and  8.  Appendix  B contains  tabulated  values  of 
Ro/0o  for  various  O.C.  curves  and  rQ.  Both  appendixes  are  modified 
from  H-108  (18). 

As  noted  by  Epstein  (7),  the  O.C.  curve  of  the  test  mih{Xr  RQ}, 

'0»'  » 

where  rQ  is  given  by  Table  4 and  n by  Equation  (5.16),  is  such  that 
L(eo)>l-a,  but  in  some  cases  L^)  may  be  slightly  greater  than  8. 

This  can  be  avoided  in  either  of  two  ways.  One  way  is  to  give  the 

experimenter  the  freedom  to  use,  instead  of  RQ,  the  slightly  larger 

2 

truncation  number  R0‘  = 0ox1_a  (2r)/n;  the  test  based  on  min{Xr  n RQ} 
will  have  L(eo)  = 1-a  and  Lfe^Bo  The  other  way  is  to  use  n+1  items 


throughout  the  test,  and  to  use,  instead  of  RQ,  the  slightly  smaller 

2 

truncation  number  RJ  1 = e x (2rJ/(n+l),  The  test  based  on 
o 0 i-a  o 

minlX^,  R- "}  will  have  L(e  ) = 1-a  and  L(e,)<e.  In  most  cases  it 
ro,n+l,  0 0 

will  make  no  difference  which  procedure  is  used. 

Sequential  Life  Tests 

As  with  the  truncated  life  test  discussed  above,  the  aim  of  the 
sequential  life  test  is  to  test  the  simple  hypothesis  H0: e=0o » against 


the  alternative  Hj:e=e  , where  el<0o,  with  Type  1 and  II  errors  equal 
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to  preassigned  values  a and  g,  respectively . Again,  n items  are  put 
on  test,  with  failed  items  repaired  or  replaced.  The  test  can  be 
terminated  either  at  failure  times  with  rejection  of  HQ,  or  at  any 
time  between  failures  with  acceptance  of  HQ.  Since  abnormally  long 
intervals  furnish  "information"  in  favor  of  HQ  and  abnormally  short 
intervals  furnish  "information"  in  favor  of  H^,  these  features  are 
not  only  reasonable  but  actually  desirable. 

A previous  development  by  Epstein  and  Sobel  (10)  is  useful  in 
this  case,  utilizing  a transformation  of  the  random  variable  from  time 
to  rounds  fired  by  the  weapon.  In  their  report  they  make  use  of 
Wald's  results  on  sequential  analysis  (20)  where  decisions  are  made 
continuously. 

During  the  life  test,  information  is  available  continuously,  so 
that  a continuous  analogue  of  the  sequential  probability  ratio  test 
of  Wald  can  be  used.  The  decision  as  the  test  is  run  depends  on 

B<(eo/ei)rexp{-(l/e1  - l/eo)V(R)}<A  (5.17) 

where  B and  A are  constants,  depending  on  a and  g,  such  that  B<1<A. 

The  decision  to  continue  experimentation  is  made  as  long  as  the  in- 
equality (5.17)  holds.  At  the  time  the  experiment  is  stopped,  if  the 
first  inequality  in  (5.17)  is  violated,  HQ  is  accepted;  if  the  second 
inequality  is  violated,  is  accepted.  As  in  Wald's  case,  the  test 
obtained  by  setting  B = e/(l-«)  and  A = (l-g)/a  is  a satisfactory 
solution  of  the  problem  from  a practical  point  of  view. 

In  Equation  (5.17),  V(R)  is  a statistic  which  can  be  interpreted 
as  the  total  number  of  rounds  fired  by  all  n guns  on  test  at  round 
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number  R,  with  R measured  from  the  beginning  of  the  test.  Clearly,  in 


the  replacement  case, 

V (R)  = nR.  (5.18) 

To  graph  the  data  continuously  versus  the  number  of  rounds  fired, 
it  is  convenient  to  write  Equation  (5.17)  in  the  form 

-hj  + rs  < V(R)  < hQ  + rs  ^ (5.19) 

where  h ,hj,  and  s are  positive  constants  given  by 

h0  = -log  B/{ (l/e1)-(l/eo)>,  (5.20) 

hj  = log  A/{(l/e1)-(l/eo)>,  (5.21) 

s = log(eo/ei)/{(l/9i-l/eo)>.  (5.22) 

Further,  as  shown  by  Wald,  the  O.C.  curve  is  given  approximately  by  a 
pair  of  parametric  equations 

L(0)  = (Ah  - l)/(Ah  - Bh)  (5.23) 

and 

e = { (0o/e1)tl  - l}/(h(l/ei  - l/eo)},  (5.24) 


by  letting  the  parameter  h run  through  all  real  values.  The  values  of 
L(e)  at  the  five  points  0=O,01,s,0o,  and  ® enable  one  to  sketch  the 
entire  curve.  These  values  are  respectively  0,0,  log  A/ (log  A-log  B), 
1-a,  and  1. 

Let  E0(r)  equal  the  expected  number  of  observations  required  to 
reach  a decision  when  0 is  the  true  parameter  value.  Since  the  loga- 
rithm of  the  middle  expression  in  Equation  (5.17)  is  either  log  B or 
log  A at  the  time  experimentation  stops,  we  have,  neglecting  only  the 
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excess  over  log  A, 

Ee(r)log(0o/01)-Ee(V{R)){l/ei-l/9o}~L(e)log  B + 


-'ll  /oU  A 

\ * “l-\u  / / iwy  r» 


It  can  be  shown  that 
E0 (V(R) ) = 0E0(r). 


(5.25) 

(5.26) 


Therefore,  from  Equations  (5.25)  and  (5.26), 

E (r)~  l(9)1o9  b * U-L(e))loq  A m hi-L(e)  (hn-%), 
9 log(o0/e1)  - e(l/01-l/eo)  s - e 


0 + s. 

(5.27) 

and 

E (r)~  -l°9  A.  lQS,  B = e = s 

9 Oog«>0/»,)>Z  IT  S> 

(5.28) 

If  k = Oq/Bj,  the  approximate  values  of  E0(r)  are 
e=01,  s,  and  e^.  They  are 

simplified  for 

E0^(r)~{elog  B + (l-e)log  A)/{log  k - (k-l)/k). 

(5.29) 

Es(r)~-log  A log  B/(log  k)2, 

(5.30) 

E0  (r)-{(l-a)log  B + ctlog  A}/{  1 og  k - (k-1)}. 
0 

(5.31) 

In  Table  5,  (page  46),  E (r)  is  given  for  6=0,  0,, 

y l * 

» s,  eQ,  »,  for 

k=3/2,  2,  5/2,  3,  and  for  o=.01,  .05,  and  s=.01,  ,05. 

The  formula  for 

L(e)  and  EQ(r),  given  by  Equations  (5.24),  (5.27)  and  (5,28)  respec- 
tively, are  approximations  to  the  actual  L(e)  and  actual  Ea(r)  arising 
from  the  use  of  the  semi-continuous  sequential  decision  rule  specified 
by  the  inequalities  (5.17).  Epstein  and  Sobel  (10)  discuss  the  accur- 
acy of  the  approximations  and  give  formulae  for  the  bounds  of  L(e)  and 
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Ee(r). 


Comparison  of  Sequential  and  Truncated  Life  Testing 

From  a purely  statistical  point  of  view,  the  sequential  test  is 
a more  efficient  test,  in  that  a decision  is  reached  in  less  time  with 
usually  a smaller  total  number  of  rounds  fired  and  with  fewer  failures 
occurring.  Figure  2 (page  48)  shows  an  example  of  the  expected  number 
of  failures  versus  e for  both  the  truncated  and  sequential  test.  As 
is  shown,  the  sequential  test  on  the  average  will  reach  a decision  with 
less  failures.  Figure  3 (page  49)  shows  an  example  of  the  expected 
number  of  rounds  fired  per  gun  for  the  sequential  and  truncated  tests. 
Again,  the  sequential  test  requires  fewer  rounds  than  the  truncated 
test,  with  the  truncated  test  requiring  considerably  more  rounds  for 
larger  values  of  0.  Since,  in  prototype  testing,  experimental  rounds 
are  used  which  can  cost  on  the  magnitude  of  $1  per  round,  the  use  of 
less  rounds  can  represent  a considerable  cash  savings.  It  also  re- 
duces the  length  of  time  needed  to  conduct  the  test,  assuming  the 
firing  rate  is  the  same  in  both  tests. 

However,  while  statistically  the  sequential  test  may  appear  the 
better  of  the  two,  practical  considerations  make  it  difficult  to  uti- 
lize. First,  the  number  of  rounds  required  for  the  test  is  not  known 
in  advance,  since  it  terminates  only  when  one  of  the  inequalities  in 
Equation  (5.17)  is  satisfied.  This  problem  can  be  avoided  by  use  of  a 
truncated  sequential  test  as  discussed  in  H-108  (18),  where  if  the 
test  continues  to  a certain  point,  a decision  is  made  on  the  basis  of 
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the  number  of  failures  which  have  occurred  up  to  that  point.  However, 
the  number  of  rounds  to  this  truncation  point  is  considerably  greater 
than  the  expected  value,  so  that  if  the  truncation  number  of  rounds  was 
ordered  by  the  testing  facility,  it  would  be  expected  that  many  of  the 
rounds  would  not  be  used,  wasting  the  money  spent  on  the  unused  rounds. 

A second  difficulty  of  implementing  this  method  is  that  all  guns 
would  have  to  be  tested  simultaneously  and  fired  at  the  same  rate. 

This  would  mean  that,  for  n guns  on  test,  n ranges  and  n riflemen  would 
be  necessary,  plus  failure  recording  personnel.  Firing  n guns  at 
exactly  the  same  rate  would  clearly  be  difficult,  plus  would  mean  that, 
when  a failure  occurs  in  any  of  the  guns,  all  weapons  would  have  to 
halt  firing  until  the  failed  gun  was  repaired. 

A third  difficulty  In  implementing  the  procedure  is  the  constant 
test  monitoring  necessary,  where  the  results  must  be  constantly  record- 
ed until  one  of  the  inequalities  In  Equation  (5.17)  is  met  and  a deci- 
sion made. 

Although  the  sequential  testing  method  would,  on  the  average,  use 
fewer  rounds  to  reach  a decision,  it  is  felt  that  additional  adminis- 
trative and  personnel  costs  would  more  than  cancel  any  savings  result- 
ing from  using  fewer  rounds.  Because  the  truncated  life  test  does  not 
have  these  difficulties,  it  is  felt  a modified  form  of  it  would  be  the 
best  testing  method.  The  proposed  procedure  is  demonstrated  with  a 
numerical  example  in  the  following  chapter. 


CHAPTER  VI 


PROPOSED  TESTING  PROCEDURE 

The  reliability  testing  procedure  recommended  by  this  report  is 
described  in  this  chapter,  with  an  example  presented  to  clarify  its 
use.  The  recommended  procedure  is  basically  the  truncated  life  test- 
ing procedure  discussed  in  the  previous  chapter,  modified  as  follows. 

Rather  than  having  all  weapons  firing  simultaneously,  it  would 
be  more  practical  to  have  the  weapons  fired  one  at  a time  for  RQ 
rounds.  Assuming  the  environment  in  which  each  weapon  is  tested  is 
the  same,  then  it  should  make  no  difference  whether  the  weapons  are 
fired  simultaneously  or  one  at  a time.  The  reason  such  a method  of 
firing  is  desirable  is  that  there  are  a limited  number  of  test  ranges 
and  personnel  available,  with  usually  2-3  personnel  performing  the 
actual  testing,  and  simultaneous  testing  would  be  quite  difficult. 
However,  individual  testing  of  the  weapons  results  in  the  additional 
test  cost  of  additional  rounds  used  if  the  actual  mean  number  of  rounds 
is  less  than  the  specified  value.  In  other  words,  the  cost  of  a reject 
decision  will  be  slightly  higher.  This  is  because  ideally  all  n wea- 

lL 

pons  would  halt  fire  when  the  r^  failure  occurs,  with  each  weapon 
saving  the  number  of  unfired  rounds  equal  to  the  difference  between 

1L 

the  round  number  R on  which  the  rQ  failure  occurs  and  RQ,  the  trun- 
cation number  of  rounds  at  which  the  test  would  have  otherwise  stopped. 
However,  when  tested  individually  each  weapon  must  be  fired  RQ  rounds 
each  until  the  rQt*1  failure  occurs  which,  if  it  does  occur,  will 
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5,?. 

probably  occur  with  the  weapon,  due  to  the  small  n on  test.  In 
other  words,  (n-l))R0-R)  rounds  will  be  wasted.  However,  if  the  mean 
number  of  rounds  between  failures  meets  or  exceeds  the  specified  num- 
ber being  tested  for,  no  rounds  are  wasted,  since  all  n guns  would 
have  fired  RQ  rounds  even  if  tested  simultaneously.  Figure  4 shows 
this  graphically  for  a particular  example. 

Testing  small  caliber  arms  amounts  to  testing  three  separate 

hypotheses.  Each  of  the  three  failure  modes  is  tested  to  determine 

whether  the  r^an  number  of  rounds  between  failures  meets  the  specified 

value.  In  other  words,  it  is  hypothesized  that  HQ: e=eo  against  an 

alternative  H.re^e.  where  0.  <e  , for  each  type  of  failure, 
li  l o r 

Let  e^c  equal  the  mean  number  of  rounds  between  immediately 
clearable  failures,  enc  equal  the  mean  number  of  rounds  between  non- 
•immediately  clearable  failures,  and  9pb  equal  the  mean  number  of  rounds 
between  part  breakages.  When  these  values  are  specified  by  the  Army, 
e-c  is  always  less  than  and  9pb,  while  epb  usually  equals  to  or  is 
greater  than  e . When  testing  the  three  hypotheses  simultaneously, 

1 1 v# 

the  largest  e being  tested,  probably  epb,  will  determine  the  value  n 
and  Rq  used  in  the  test,  since  these  numbers  would  be  smaller  for  test- 
for  a smaller  9. 

In  conducting  the  test,  when  testing  for  the  largest  e for  o and 
3 fixed,  more  rounds  than  necessary  are  fired  to  test  a lower  0 at  the 
same  levels  of  a and  3=  If  the  test  for  the  lower  value  of  e uses  ail 
Rq  rounds  fired  in  testing  for  the  larger  e,  than  the  a and  3 levels 
for  the  test  of  the  smaller  e will  be  larger  than  those  set  for  testing 
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the  larger  e.  The  following  numerical  example  should  clarify  these 
comments. 

Suppose  an  Army  specification  on  a prototype  small  caliber  arm 
is  as  follows: 

(1)  Malfunction  clearable  by  immediate  action  not  to  exceed  1 
per  1000  rounds,  with  o<_.10,  e<.10. 

[?,)  Malfunction  not  clearable  by  immediate  action  but  not  re- 
quiring parts  replacement  not  to  exceed  1 per  6000  rounds,  with  o=.10, 

S=.10,  e /e  =3. 

O 1 

(3)  Malfunction  requiring  parts  replacement  not  to  exceed  1 per 
6000  rounds,  with  a=.10,  B=.10,  *q/b  *3, 

This  is  the  general  form  of  specification  suggested  by  this  report.  It 
is  noticeably  different  from  the  form  presently  used  due  to  the  inclu- 
sion of  a, e,  and  e /e,  values. 

0 1 

The  specification  stipulates  0^C=1OOO  rounds,  ^^pb^OOO  rounds, 
for  a<_.10,  B<_.10,  0q/01=3o  Suppose  10  prototypes  are  delivered  from 

each  contractor  and  let  all  10  be  used  in  the  test,  fixing  n=10. 

2 2 

To  find  r.  , find  the  smallest  r such  that  x (2r)/xo(2r)>0  /0  . 
o i-a  3 “ i 0 

In  this  case,  this  is 
x\  (2r) 

— >1/3.  (6.1) 

x\  (2r) 

Using  tables  of  the  chi-square  distribution  gives 
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x 9 (2r)/x  i(2r) 
r _J « 

1 .0458 

3 .207 

5 .3043 

6 .3398 

which  gives  ro=6„  With  n=10,  0=6000,  and  using  Equation  (5.16)  gives 
10  = {18912/iy 

which  sets  RQ=1891  rounds.  Therefore,  for  testing  both  epb  and  enc, 

this  gives  rQ=6,  n=10,  RQ=1891  rounds  for  the  a,e,  and  0o/0j.  specified. 
However,  it  should  be  understood  that  the  rQ=6.1s  for  testing  epb  and 
0n.  only,  not  0.  . Each  0 has  its  own  associated  r , and  r is  the 

lit*  IU  0 0 

same  for  both  0„.  and  e„„  only  because  0„k=0„,.o  If  it  is  desired 
pb  nc  J pb  nc 

that  a=0l  and  e=.l  for  testing  for  0.  as  well,  then  use  of  Equation 

I v 

(5.16)  gives. 

10  = {1000  (x2  (2r)/(2)(1891)} 

.9  0 

giving 

X*9  (2r0)  <37.82 

which  gives  r0=25„  However,  use  of  such  a large  rQ  "wastes"  informa- 
tion, since  a lower  value  could  be  used  in  exchange  for  either  smaller 
a and  e levels,  or  else  a smaller  value  of  0/0  for  the  same  a and 
6 levels,  which  increases  the  discriminating  ability  of  the  test. 
However,  using  this  value  for  r , the  test  procedure  would  be  as 
follows: 

(1)  Each  of  the  n=10  guns  is  fired  one  at  a time  for  RQ=1891 
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rounds.  When  a failure  occurs,  the  round  number,  failure  number,  and 
type  of  failure  is  recorded. 

(2)  Continue  testing  until  25  immediately  clearable  failures,  6 
non-immediately  clearable  failures,  and  6 parts  replacements  occur,  or 
else  until  all  weapons  have  fired  RQ=1891  rounds. 

(3)  If  all  the  weapons  fired  1891  rounds  without  the  above  num- 
ber of  failures  occurring,  the  three  hypotheses  are  accepted  at  the 
specified  levels  of  a,$,  and  eo/e x . That  is,  the  weapon  meets  its 
specifications.  If  any  of  the  failure  numbers  given  above  is  reached 
before  all  weapons  have  fired  1891  rounds,  the  hypothesis  associated 
with  the  particular  failure  mode  is  rejected,  and  the  weapon  does  not 
completely  meet  the  specifications.  If  all  of  the  above  failure  num- 
bers are  exceeded  before  all  guns  have  fired  1891  rounds,  all  three 
hypotheses  are  rejected,  and  the  weapon  doesn't  meet  any  of  the  speci- 
fications. 

This  example  illustrates  the  application  of  this  type  of  test. 

The  testing  procedure  has  both  its  advantages  and  disadvantages,  as 
discussed  in  the  following  conclusion. 


1 
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CHAPTER  VII  j 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  report  has  attempted  to  improve  on  present  techniques  used 
by  the  U,S.  Army  to  test  for  the  reliability  of  prototype  small  cali- 
ber arms  submitted  for  engineering  design  tests  by  several  private 
contractors.  The  present  methods  used  by  the  Army  are  rather  arbi- 
trary, varying  in  method  from  test  to  test,  and  are  not  built  on  a 
sound  statistical  basis.  Present  techniques  fail  to  quantify  in 
statistical  terms  either  the  specifications  nor  the  testing  for  the 
actual  values. 

Essentially,  the  random  nature  of  the  specified  parameters  is 
ignored  and  no  measure  is  provided  as  to  whether  the  desired  minimum 
failure  rates  are  actually  being  achieved.  Failure  to  achieve  a 
desired  mean  life  in  weaponry  of  this  type  will  have  a tremendous  im-  j 

pact  on  the  overall  life  cycle  costs  of  the  weapon  in  the  form  of  j 

increased  repair  and  maintenance,  availability,  downtime,  and  overall  j 

effectiveness.  For  this  reason.it  Is  felt  that  stringent  test  proce-  ] 

dures  in  the  prototype  stage  of  development  are  of  extreme  importance.  J 

j 

The  proposed  reliability  test  procedure  developed  in  this  report  | 

is  of  much  greater  efficiency  than  past  methods  used  in  the  area  of 
armament  testing,  and  is  an  attempt  to  standardize  testing  techniques 
in  this  area  as  well  as  provide  a measure  of  how  much  confidence  can 
be  placed  in  the  results  of  the  reliability  test.  Among  the  proposals 
presented  in  this  report  are: 
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(1)  More  complete  specifications  of  the  reliability  requirement 
of  a proposed  prototype  gun  in  the  contract  stage  of  development.  Be- 
sides stating  what  failure  rate  is  desired  for  each  failure  mode,  such 
quantities  as  a,B,  and  6o/ei  .levels  should  be  specified  to  help  insure 
that  what  is  specified  is  the  same  as  what  is  achieved. 

(2)  Changing  the  testing  procedure  to  the  procedure  described  in 
this  report.  This  would  determine  whether  the  weapons  actually  meet 
the  specifications,  and  help  to  minimize  overall  life  cycle  costs 
which  result  from  acceptance  of  substandard  items. 

The  testing  procedure  described  in  this  report  does  have  one 
major  drawback.  The  assumption  of  an  underlying  exponential  distri- 
bution of  the  number  of  rounds  between  failures  for  each  failure  mode 
could  prove  to  be  a major  restriction.  Analysis  of  past  firing  data 
is  needed  to  determine  the  actual  underlying  distribution  of  the 
number  of  rounds  to  failure  for  each  failure  mode.  Also  needed  is  a 
testing  plan  which  allows  for  testing  with  replacement  when  the  assumed 
underlying  probability  distribution  is  Wei bull  or  gamma,  which  would 
probably  have. greater  accuracy  and  flexibility  than  the  exponential 
distribution,  which  is  a special  case  of  both.  Any  application  of  the 
proposed  testing  plan  presented  in  this  report  would  yield  invalid 
results  if  the  underlying  assumptions  are  not  met. 

The  effect  of  using  a test  which  assumes  one  type  of  distribution 
when  actually  it  is  of  a different  type  has  been  researched  to  a de- 
gree. Statistical  procedures  which  are  relatively  insensitive  to 
departures  from  the  assumed  form  of  the  distribution  are  called  robust. 
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Oannemll ler  and  Zelen  (3)  have  Investigated  the  effect  of  utilizing 
testing  procedures  which  were  based  on  the  assumption  of  an  exponen- 
tial distribution  when  the  underlying  distribution  was  actually  Wei- 
bull  with  shape  parameter  2.  They  conclude  that  none  of  the  test  pro- 
cedures are  Insensitive  to  departures  from  the  assumed  distribution. 

However(  they  also  conclude  that  a procedure  of  the  type  proposed  here 
Is  on  the  conservative  side;  that  1s»  the  6 level  Indicated  may  be  a 
conservative  figure.  However,  from  the  view  of  the  producer,  this  may 
not  be  desirable. 

It  Is  felt,  however,  that  the  proposed  testing  plan  Is  at  the 
least  an  effort  In  the  direction  of  better  measurement  of  the  relia- 
bility of  a prototype  weapon,  and  makes  better  use  of  available  Infor- 
mation than  those -methods  now  used. 
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APPENDIX  A 

Tables  for  Truncated  Life  Testing 

The  following  tables  give  values  of  n and  r for  values  of 
WW®’  an<*  e for  a truncated  testing  procedure  with  replace 
ment.  Tables  are  modified  from  H-108  (18). 


APPENDIX  B 


Tables  for  Truncated  Life  Testing 

The  following  tables  give  values  of  RQ/eo  for  the  various  O.C. 
curves  and  rQ  given.  The  code  number  given  in  the  tables  refers  to 
the  corresponding  O.C.  curve  having  the  same  code  number.  Tables  and 
figures  are  modified  from  H-108  (18). 
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